1' 



® 



J 



Europaisches Paten tarn t 
European Patent Office 
Office europeen des brevets 





(ft) Publication number : 0 687 1 06 A1 



@ 



EUROPEAN PATENT APPLICATION 



21) Application number : 95303822.1 
S) Date of filing : 05.06.95 



©Int. Cl. 6 : H04N 5/217 



@) Priority: 06.06.94 JP 123886/94 

@ Date of publication of application : 
13.12.95 Bulletin 95/50 



@) Designated Contracting States : 
DE FR GB 

(3) Applicant: MATSUSHITA ELECTRIC 
INDUSTRIAL CO., LTD. 
1006, Ohaza Kadoma 
Kadoma-shi, Osaka 571 (JP) 



(72) Inventor : Tabei, Kenji 
Saedoniryo 575, 
749-1, Saedocho, 
Tsuzuki-ku 
Yokohama-shi (JP) 
Inventor: Nishizawa, Masato 
11-204, Wakabadai-3-chome, 
Asahi-ku 

Yokohama-shi (JP) 

@) Representative : Votier, Sidney David 
CARPMAELS & RANSFORD 
43, Bloomsbury Square 
London WC1A 2RA (GB) 



(g) Defective pixel correction circuit 



< 

CO 

CO 
CO 



@ A defective pixel correction circuit corrects a 
defective pixel in a solid imaging device such as 
a CCD exactly and sufficiently. A boundary 
detection circuit (11) calculates magnitudes of 
boundaries from signals of eight peripheral 
pixels taken in a pixel taking-in circuit (10) and a 
boundary ordering circuit (12) compares the 
calculated magnitudes of the boundaries with 
one another to order the magnitudes of the 
boundaries. An interpolation circuit (14) pro- 
duces an interpolation signal in accordance 
with an interpolation method determined by an 
interpolation method determining circuit (13) 
on the basis of the order of the ordered bound- 
aries to correct the defective pixel. 
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BACKGROUND OF THE INVENTION 
FIELD OF THE INVENTION 

5 The present invention relates to a defective pixel correction circuit for correcting a defective pixel in a solid 

imaging device such as a CCD. 

DESCRIPTION OF THE PRIOR ART 

10 Correction of a defective pixel in a CCD or the like is heretofore performed by a so-called 0-order hold 

interpolation method in which positional information of a defective pixel is previously stored and the defective 
pixel is simply replaced by information obtained from a pixel adjacent to the defective pixel on the basis of the 
positional information. In this correction method, however, as shown in Fig. 1 , when a boundary between light 
and dark portions of an image is present just before a defective pixel, for example, the place X in which the 

15 defective pixel is corrected is conspicuous. 

In order to solve such a problem, JP-A-3-296374 proposes a correction method in which levels of image 
signals produced from a plurality of pixels in the vicinity of a defective pixel are compared with one another to 
select image information for the pixel used for interpolation on the basis of the comparison result so that image 
information at the position of the defective pixel is interpolated. The patent publication as mentioned above 

20 discloses the following measures. 

More particularly, as shown in Fig. 2, absolute values of signal level differences between adjacent pixels 
(boundaries I, II, III and IV) for five pixels including pixels D and E disposed before and after and adjacent to 
a defective pixel X and pixels A, B and C disposed before one horizontal line are calculated. 

The absolute values of the output signal level differences between the adjacent pixels are compared be- 

25 tween the boundaries I and II, I and III, IV and II, and IV and III to produce four kinds of control signals a to d 
determined in accordance with Table 1 on the basis of the comparison among the four boundaries. 
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As shown in Figs. 3Aand 3B, the control signals a to d correspond to four kinds of stripe patterns including 
a horizontal pattern shown in Fig. 3A, a right tilted pattern shown in Fig. 3B, a left tilted pattern shown in Fig. 
2C and a vertical pattern shown in Fig. 3D. The defective pixel is replaced by an average of the output data 
of the pixels D and E for the horizontal pattern of Fig. 3A, the output data of the pixel E for the right tilted pattern 
of Fig. 3B, the output data of the pixel D for the left tilted pattern of Fig. 3C and the output data of the pixel B 
for the vertical pattern of Fig. 3D to be corrected. 

In the correction circuit proposed in the above-mentioned JP-A-3-296374, however, the absolute values 
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of the level differences between the pixels are calculated from the levels of the Image signals produced from 
the five pixels including the right and left pixels of the defective pixel and the three pixels disposed before one 
line of the defective pixel and are compared with one another to thereby identify a pattern of an object to be 
imaged in the vicinity of the defective pixel and accordingly the pattern is sometimes identified in error. 
5 For example, in Fig. 4A, when outputs of peripheral pixels are A=100, B=120, C=200, D=110 and E=215 f 

absolute values of boundaries are calculated as follows: 

Boundary I = I A - D| =10 
Boundary II = I B - A I =20 
Boundary III = |C- B| =80 
10 Boundary IV = |E- c| =15 

Accordingly, this case corresponds to No. 14 of Table 1 and the control signal d is selected. However, since 
the pattern of Fig. 4A is tilted right, the pattern must be identified as the right tilted pattern of Fig. 3B and the 
control signal b must be selected. Further, in Fig. 4B, when outputs of peripheral pixels are A=100, B=110, 
C=200, D=120 and E=205, absolute values of boundaries are calculated as follows: 
15 Boundary I = I A - D I =20 

Boundary II = |B-A| = 10 
Boundary III = |C- B| =90 
Boundary IV = I E - C I =5 
Accordingly, this case corresponds to No. 7 of Table 1 and the control signal a is selected. However, since the 
20 pattern of Fig. 4B is vertical, the pattern must be identified as the vertical pattern of Fig. 3D and the control 
signal d must be selected. 

As apparent from the above two examples, there is a problem that the correction circuit cannot identify 
the pattern at the boundary in the vicinity of the defective pixel exactly. Further, since the patterns of an object 
to be identified are limited to four kinds of patterns including the horizontal, left tilted, right tilted and vertical 
25 patterns, sufficient and fine correction is impossible. 

SUMMARY OF THE INVENTION 

It is an object of the present invention to solve the above problems in the prior art by providing an excellent 

30 defective pixel correction circuit capable of correcting a defective pixel exactly even when a complicated bound- 
ary is present in an object to be imaged in the vicinity of the defective pixel. 

In order to achieve the above object, the defective pixel correction circuit according to the present invention 
comprises a pixel taking-in circuit for taking in signals produced from pixels in the vicinity of a defective pixel 
supplied from a digital image, a boundary detection circuit for calculating magnitudes of boundaries between 

35 peripheral adjacent pixels from the signals produced from the taken-in pixels, a boundary ordering circuit for 
comparing the magnitudes of the detected boundaries to order the boundaries, an interpolation method deter- 
mining circuit for determining an interpolation method on the basis of inputs from the boundary ordering circuit, 
and an interpolation circuit for producing an interpolation output in accordance with the interpolation method 
determined by said interpolation method determining circuit. 

40 With the above conf iguration, according to the present invention, the magnitudes of the boundaries be- 

tween the peripheral pixels are compared with one another to order the magnitudes of the boundaries so that 
even when a complicated boundary is present in the vicinity of the defective pixel, the pattern of the boundary 
can be identified. Further, the interpolation method determining circuit previously determines the-optimum in- 
terpolation method for each identified pattern so that correction of the defective pixel can be made exactly for 

45 each boundary pattern. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a diagram for explaining a conventional correction process using a 0-order hold interpolation; 
so Fig. 2 shows a pixel arrangement in the vicinity of a defective pixel in a prior art 

Figs. 3A to 3D show stripe patterns identified in a prior art; 
Figs. 4A and 4B show patterns which cannot be identified in a prior art; 

Fig. 5 is a block diagram schematically illustrating an embodiment of a defective pixel correction circuit of 
the present invention; 
55 Fig. 6 shows a window of pixels taken in by a pixel taking-in circuit; 

Fig. 7 shows a window of pixels taken in a pixel taking-in circuit in a frame image in the interlaced scanning 
system; 

Fig. 8 shows positions of boundaries; 
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Fig. 9 shows an actual example of detected boundaries; 

Figs. 1 0A and 10B show examples of actual boundaries presumed from detected boundaries; 
Figs. 11Aand 11B show examples of interpolation methods suitable for boundary patterns; and 
Figs. 12Ato 12L show 12 kinds of interpolation methods. 

5 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Embodiments of the present invention are now described with reference to the accompanying drawings. 
Fig. 5 is a block diagram schematically illustrating an embodiment of the present invention. Numeral 10 

w denotes a pixel taking-in circuit for taking in signals of nine pixels corresponding to a window of 3x3 supplied 
from a digital imaging device such as a CCD, numeral 11 denotes a boundary detection circuit for calculating 
magnitudes of boundaries between adjacent pixels on the basis of eight peripheral or surrounding pixels taken 
in by the pixel taking-in circuit 10, numeral 12 denotes a boundary ordering circuit for ordering the boundaries 
on the basis of the magnitudes of the boundaries calculated by the boundary detection circuit 11 f numeral 13 

15 denotes an interpolation method determining circuit for determining an interpolation method on the basis of an 
output of the boundary ordering circuit 12, and numeral 14 denotes an interpolation circuit for performing in- 
terpolation using signals from the eight surrounding pixels taken in by the pixel taking-in circuit 10 in accordance 
with the interpolation method determined by the interpolation method determining circuit 13. 

Numeral 15 denotes a memory for storing a position of a defective pixel, numeral 16 denotes a system 

20 controller for judging whether the position of the center pixel taken in by the pixel taking-in circuit 1 0 is stored 
in the memory 1 5 or not to control a change-over circuit 1 7, and numeral 1 7 denotes the changing-over circuit 
for selecting between a signal of the center pixel taken in by the pixel taking-in circuit 10 and an Interpolation 
output from the interpolation circuit 14 in accordance with a control signal from the system controller 16. 
Operation of the embodiment is now described. 

25 A method of taking in an image is first described. The pixel taking-in circuit 10 takes in the signals of nine 

pixels corresponding to the window of 3x3 and produces the signals. That is, as shown in Fig. 6, the pixel taking- 
in circuit 1 0 produces a signal x11 from X11 corresponding to the center pixel and signals xOO to x02, x10, x12 
and x20 to x22 from XOO to X02, X1 0, X1 2 and X20 to X22 corresponding to the surrounding or peripheral pixels, 
respectively. However, in the case of an interlaced signal in the interlaced scanning method, interpolation is 

30 made in a field and accordingly the nine pixels correspond to a window of 3x3 in the field. In other words, in 
the case of the interlaced scanning, the pixel taking-in circuit takes in signals from pixels disposed before two 
lines and after two lines in a frame picture as shown in Fig. 7. 

• . . . -• . *■ _ _i i _l! w^v. .r+A^-lt-Kr* k«ti»,ofin tko enrTAiinHlnn rvvolc is now HARririhftri. 

/\ meillUU Ul UttlUUIcUIII^ lllcayi IllUUCd Ul mo uwunuun^ i/ymwvu w«-.. ...... r — 

The boundary detection circuit 11 calculates absolute values of differences between the signals from adjacent 
35 pixels of the eight surrounding pixels taken in by the pixel taking-in circuit 10 as shown in Fig. 8. However, the 
magnitudes of the boundaries between the vertically adjacent pixels are multiplied by a coefficient k. That is, 
the magnitudes d1 to d8 of the boundaries are calculated by the following equation 1: 

Equation 1: d1 = k - 1x10 - x00| 

d2 = |x00 - x0l| 

40 d3 = 1x01 - x02| 

d4 = k - |x02 - x12| 
d5 = k - |x12 - x22| 
d6 = |x22 - x2l| 

d7 = 1x21 - x20l 

45 d8 = k - |x20 - x10| 

where the coefficient k is a positive real number which is used to correct the strength of the con-elation between 
pixels in the horizontal and vertical directions caused by different distances between pixels for calculation of 
a signal difference in the horizontal and vertical direction due to the aspect ratio of the image and the interlaced 
scanning. 

so A method of comparing the boundaries in magnitude is now described. The boundary ordering circuit 12 

compares the magnitudes d1 to d8 of the boundaries with one another with respect to all combinations of 56 

sets including d1 and d2, d1 and d3 d7 and d8 and produces the order of the boundary magnitude d1 as 

D1, the order of d2 as D2, and the orders of the boundary magnitude d3 to d8 as D3 to D8, respectively. A 
combination of D1 to D8 is named a boundary order pattern. Further, when the comparison results are equal 
55 to each other, the relation in magnitude is determined in accordance with the following rules: 

Rule 1 : when the comparison results in magnitude of the boundaries in the vertical and horizontal directions 
are equal, it is assumed that the magnitude of the boundary having smaller one of the con-elation between 
the horizontally adjacent pixels and the correlation between the vertically adjacent pixels is larger than 
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that of the other. Further, in the interlaced scanning system, since the correlation between the horizontally 
adjacent pixels is generally higher than the correlation between the vertically adjacent pixels, the boundary 
in the vertical direction is assumed to be larger in magnitude when the comparison results in magnitude 
of the boundaries in the vertical and horizontal directions are equal. 
5 Rule 2: when the comparison results in magnitude of the boundaries in the same direction are equal, va- 

rious rules can be considered since the positional relation is symmetrical, while the boundary having a 
smaller suffix is assumed to be larger than the boundary having a larger suffix in the embodiment. 
The rules 1 and 2 are combined to form the relation of the following equation 2 and accordingly when the 
comparison results are equal, the boundaries are ordered in accordance with the equation 2: 
10 Equation 2: d1 > d4 > d5 > d8 > d2 > d3 > d6 > 67 

The above operation is now described with reference to a definite example. When the relation of d1 to d8 
is defined by the following equation 3, the boundary order pattern is calculated. 

Equation 3: d3 > d5 > d1 > d6 > d8 > d2 = 67 > d4 
That is, d1 is in a third place and accordingly the order D1 of d1 is three. Further, d2 and 67 are both in 
15 the sixth place, while since it is judged that d2 is larger than d7 (d2 > d7) in accordance with the above rule, 
d2 is in the sixth place and d7 is the seventh place. Accordingly, D1 to D8 are determined to be the following 
equation 4: 

Equation 4: D1 = 3, D2 = 6, D3 = 1, D4 = 8, D5 = 2, D6 = 4, D7 = 7, D8 = 5 
A method of determining an interpolation method from the comparison results is now described. The 
20 boundary order pattern produced by the boundary ordering circuit 12 represents a combined order of the mag- 
nitudes of the eight boundaries and accordingly there are eight factorial, that is, 8!=40320 kinds of patterns. 
The interpolation method determining circuit 1 3 assigns a peculiar number to each of the boundary order pat- 
terns produced by the boundary ordering circuit 12. This number is named the boundary order pattern number. 
The boundary order pattern number is calculated from the boundary order pattern D1 to D8 produced by the 
25 boundary ordering circuit 12 in accordance with the following equation 5. 

Equation 5: PN = D1 + 8(D2 + 8(D3 + 8(D4 + 8(D5 + 8(D6 + 8(D7 + 8 D8)))))) 

Since D1 to DB do not have the same value, the pattern number PN does not have a continuous number. 

In order to determine the interpolation method for the boundary order pattern classified above, the inter- 
polation method determining method 1 3 uniquely sets the interpolation method to be outputted for each of the 
30 patterns previously. The relation of the order of the boundary magnitude in the comparison result and the in- 
terpolation method is now described definitely. 

When it is assumed that the boundary order pattern D1 to D8 produced by the boundary ordering circuit 
12 is set as indicated by the following equation 6 and the order of the boundary magnitude is set as shown in 
Fig. 9, boundaries of an actual peripheral image as shown in Figs. 1 0A and 10B are considered as boundaries 
35 of the actual peripheral image to be presumed from the boundaries shown in Fig. 9. 

Equation 6: D1 = 7, D2 = 8, D3 = 1, D4 = 5, D5 = 4, D6 = 3, D7 = 6, D8 = 2 
Thus, when the patterns of the peripheral image are different, optimum interpolation methods therefor are also 
different. The interpolation method of Fig. 11 A is considered to be optimum for the pattern of the peripheral 
image of Fig. 10Aandthe interpolation method of Fig. 11 B is considered to be optimum for the pattern of Fig. 
40 10B. 

However, when the boundary order pattern as shown in Fig. 9 is obtained by the boundary ordering circuit 
12, it is difficult to uniquely determine the optimum interpolation method from a large number of candidates 
for the interpolation methods including those as shown in Figs. 11A and 11 B. Accordingly, a simulation is pre- 
viously made in accordance with the following method to determine an optimum interpolation method for each 
45 of the boundary order patterns. 

The simulation uses a digital image having no defective pixel. 

First of all, nine pixels corresponding to a window of 3x3 having a center of coordinates (x,y) are taken in 
from the image. In order to specify the process for the window of 3x3 having the center of coordinates (x,y), 
a value of the center pixel is attached with a suffix of (x,y) such as x1 1(x,y). Further, in the case of the interlaced 
so scanning system, pixels of the window of 3x3 for a field image are taken in the same manner as in the pixel 
taking-in circuit 10. 

The nine pixels taken in by the circuit 1 0 are processed by the boundary detection circuit 1 1 and the bound- 
ary ordering circuit 12 as described above so that the boundary order patterns D1 (x,y) to D8(x,y) are calculated 
and the boundary order pattern number PN(x,y) is calculated in accordance with the equation 5. 
55 Interpolation outputs interpolated by 12 kinds of interpolation methods shown in Figs. 12A to 12L are cal- 

culated from the eight surrounding pixels taken in by the circuit 10 so that an interpolation output calculated 
by an interpolation method 1 is assumed to y1(x,y), an interpolation output by an interpolation method 2 is as- 
sumed to y2(x,y) and interpolation outputs by interpolation methods 3 to 12 are assumed to y3(x,y) to y12(x,y). 
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respectively. Figs. 12Ato 12L show the 12kinds of simple interpolation methods, while other interpolation meth- 
ods using an average or a weighted average of a plurality of pixels may be considered. Generally, an interpo- 
lation output yn can be expressed by the following equation 7: 
Equation 7: yn = alxOO + a2 x01 + a3x02 + a4 x12 + a5x22 + a 6 x21 + a7 x20 + a8-x10 

5 where a1 to a8 represent coefficients of actual numbers. 

On the other hand, in this simulation, pixels are taken in from an image having no defective pixel and ac- 
cordingly a signal x11(x,y) produced by a center pixel X11(x,y) in the window of 3x3 can be considered to be 
a target value for correction of the defective pixel. Thus, interpolation errors of the respective interpolation 
methods can be calculated. An interpolation error ERRi(x,y) of an interpolation method i is given by the following 

10 equation 8: 

Equation 8: ERRi(x,y) = lyi(x,y) - x11(x,y) | 

where i = 1,2 12 

The interpolation error ERRi(x,y) is calculated for the whole image and a squared average of the error is 
calculated for each boundary pattern number. That is, when an interpolation method i is adopted in the bound- 
15 ary order pattern number j, a squared average TOTALERR(ij) of the interpolation error is given by the following 
equation 9: 

Equation 9: TOTALERR(i.j) = SHj(x,y) ERRi(x f y) ERRi(x,y) (x,y) e all pixels 

where 

Hj(x,y)=1:(forj=PN(x,y)) 

20 Hj(x,y) = 0: (for j*PN(x,y)) 

By selecting the interpolation method having the minimum squared average of the squared averages TO- 

TALERR(1,j), TOTALERR(2j) TOTALERR(1 2 j) of the interpolation errors by the 12 kinds of interpolation 

methods, the optimum interpolation method can be determined uniquely for the boundary order pattern number 
j. The above operation is made for all of the boundary order pattern numbers. 

25 Even when it is difficult to uniquely determine the interpolation method suitable for the boundary pattern 

obtained from only the surrounding pixels by means of intuitive judgement, the simulation can be performed 
for i mage data having various conditions to thereby determine the interpolation method suitable for each bound- 
ary pattern. 

The interpolation method is now described. The interpolation circuit 14 performs calculation for signals pro- 
30 duced from the eight surrounding pixels in accordance with the 12 kinds of interpolation methods (equation 
10 described later) shown in Figs. 12Ato 12L to produce interpolation signals y1 to y12. An interpolation signal 
of the interpolation method determined by the interpolation method determining circuit 13 is selected from the 
above 12 kinds of interpolation signals and is produced. The interpolation method expressed by the equation 
7 can be added as the interpolation method to be selected except the 1 2 kinds of interpolation methods shown 
35 in Figs. 12Ato 12L. 

Equation 10: Interpolation Method 1: y1 = x10 
Interpolation Method 2: y2 = x12 
Interpolation Method 3: y3 = x01 
Interpolation Method 4: y4 = x21 
40 Interpolation Method 5: y5 = (x10 + x12)/2 

Interpolation Method 6: y6 = (x01 + x21)/2 
Interpolation Method 7: y7 = (x00 + x22)/2 
Interpolation Method 8: y8 = (x02 + x20)/2 
Interpolation Method 9: y9 = xOO 
45 Interpolation Method 10: y10 = x02 

Interpolation Method 11: y11 = x22 
Interpolation Method 12:y12 = x20 
Finally, the memory 15, the system controller 16 and the changing-over circuit 17 are described. 
The memory 15 stores data indicative of positions of a plurality of defective pixels contained in a solid im- 
so aging device such as a CCD. The system controller 1 6 reads out the position data of the defective pixels from 
the memory 15 and controls the changing-over circuit 17 to produce the interpolation output when the position 
of the center pixel of the window of 3x3 taken in by the pixel taking-in circuit 10 is coincident with the position 
of the defective pixel stored in the memory 15 and to produce the center pixel x11 when not coincident 

As described above, according to the defective pixel correction circuit of the present invention, the mag- 
55 nitudes of the boundaries between the peripheral or surrounding pixels are ordered to thereby identify the 
boundary pattern in the vicinity of the defective pixel so that the interpolation method suitable for the pattern 
is selected to perform interpolation. Accordingly, even when the complicated boundary is present in the vicinity 
of the defective pixel, the boundary can be identified to correct the defective pixel finely and exactly. 

6 
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Claims 

1. A defective pixel correction circuit comprising: 

a pixel taking-in circuit (10) for taking in signals produced from peripheral pixels of a defective pixel 
in an imaging device; 

a boundary detection circuit (11) for detecting magnitudes of boundaries between adjacent pixels 
of the taken-ih peripheral pixels; 

a boundary ordering circuit (12) using outputs of said boundary detection circuit for ordering the 
boundaries; 

an interpolation method determining circuit (13) for determining an interpolation method on the ba- 
sis of outputs of said boundary ordering circuit, and 

an interpolation circuit (14) for producing an interpolation signal by said determined interpolation 
method. 

2. A defective pixel correction circuit comprising: 

a pixel taking-in circuit (10) for taking in signals produced from peripheral pixels of a defective pixel 
in an imaging device; 

a boundary detection circuit (11) for detecting magnitudes of boundaries between adjacent pixels 
of the taken-in peripheral pixels; 

a boundary ordering circuit (12) using outputs of said boundary detection circuit for ordering the 
boundaries; 

an interpolation method determining circuit (13) for determining an interpolation method on the ba- 
sis of outputs of said boundary ordering circuit; 

an interpolation circuit (14) for producing an interpolation signal by said determined interpolation 
method; 

a memory (15) for storing a position of said defective pixel; and 

replacement control means (17) forjudging whether data is stored in said memory or not and re- 
placing an image of said pixel taking-in circuit with an image of said interpolation circuit when data is stored 
in said memory. 

3. A defective pixel correction circuit according to Claim 1, wherein said boundary detection circuit is to cal- 
culate the magnitudes of the boundaries between the adjacent pixels from eight peripheral pixels in said 
pixel taking-in circuit and said boundary ordering circuit is to effect determination on the basis of the mag- 
nitudes of the boundaries between the pixels. 

4. A defective pixel correction circuit according to Claim 2, wherein said boundary detection circuit is to cal- 
culate the magnitudes of the boundaries between the adjacent pixels from eight peripheral pixels in said 
pixel taking-in circuit and said boundary ordering circuit is to effect determination on the basis of the mag- 
nitudes of the boundaries between the pixels. 

5. A defective pixel correction circuit according to Claim 3 or 4, wherein said boundary ordering circuit in- 
cludes a rule forjudging that the magnitude of the boundary in the vertical direction is larger than that in 
the horizontal direction when the magnitudes of the boundaries in the vertical and horizontal directions 
are equal to each other. 

6. A defective pixel correction circuit according to Claim 3 or 4, wherein said boundary ordering circuit in- 
cludes a rule for judging that the magnitude of the boundary having a smaller suffix is larger than that 
having a larger suffix when the magnitudes of the boundaries in the vertical direction or the horizontal 
direction are equal to each other. 

7. A defective pixel correction circuit according to Claim 3 or 4, wherein said boundary ordering circuit in- 
cludes a rule forjudging that the magnitude of the boundary in the vertical direction is larger than that in 
the horizontal direction when the magnitude of the boundary in the vertical direction is equal to that in 
the horizontal direction and judging that the magnitude of the boundary having a smaller suffix is larger 
than that having a larger suffix when the magnitudes of the boundaries in the vertical direction or the hor- 
izontal direction are equal to each other. 
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FIG.5 
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